Peer Review for Journals: Evidence on Quality Control, Fairness, and Innovation

I reviewed the published empirical evidence concerning journal peer review, which consisted of 68 papers, all but three published since 1975. Peer review improves quality, but its use to screen papers has met with limited success. Current procedures to assure quality and fairness seem to discourage scientific advancement, especially important innovations, because findings that conflict with current beliefs are often judged to have defects. Editors can use procedures to encourage the publication of papers with innovative findings such as invited papers, early-acceptance procedures, author nominations of reviewers, results-blind reviews, structured rating sheets, open peer review, and, in particular, electronic publication. Some journals are currently using these procedures. The basic principle behind the proposals is to change the decision from whether to publish a paper to how to publish it. Comments Postprint version. Published in Science and Engineering Ethics, Volume 3, Issue 1, January 1997, pages 63-84. Publisher URL: http://www.opragen.co.uk/SEE/contents.php3 This journal article is available at ScholarlyCommons: http://repository.upenn.edu/marketing_papers/104 Published in Science and Engineering Ethics, 3 (1997), pp. 63-84 Peer Review for Journals: Evidence on Quality Control, Fairness, and Innovation J. Scott Armstrong The Wharton School, University of Pennsylvania Philadelphia, PA Abstract I reviewed the published empirical evidence concerning journal peer review, which consisted of 68 papers, all but three published since 1975. Peer review improves quality, but its use to screen papers has met with limited success. Current procedures to assure quality and fairness seem to discourage scientific advancement, especially important innovations, because findings that conflict with current beliefs are often judged to have defects. Editors can use procedures to encourage the publication of papers with innovative findings such as invited papers, early-acceptance procedures, author nominations of reviewers, results-blind reviews, structured rating sheets, open peer review, and, in particular, electronic publication. Some journals are currently using these procedures. The basic principle behind the proposals is to change the decision from whether to publish a paper to how to publish itI reviewed the published empirical evidence concerning journal peer review, which consisted of 68 papers, all but three published since 1975. Peer review improves quality, but its use to screen papers has met with limited success. Current procedures to assure quality and fairness seem to discourage scientific advancement, especially important innovations, because findings that conflict with current beliefs are often judged to have defects. Editors can use procedures to encourage the publication of papers with innovative findings such as invited papers, early-acceptance procedures, author nominations of reviewers, results-blind reviews, structured rating sheets, open peer review, and, in particular, electronic publication. Some journals are currently using these procedures. The basic principle behind the proposals is to change the decision from whether to publish a paper to how to publish it Acknowledgements: Many people provided useful comments. Among them were Janice M. Beyer, Fred Collopy, Willam M. Epstein, Richard H. Franke, Raymond Hubbard, Joel Kupfersmid, Byron Sharp, Arthur E. Stamps III, Brian Wansink, David Watkins, and anonymous reviewers. This does not imply that they agreed with my conclusions. Mary Haight and Dara Yang provided editorial assistance. The author, a professor at the Wharton School since 1968, is a founder of two journals and has been on the editorial boards of 14 journals. Address for correspondence J. Scott Armstrong Marketing Department The Wharton School University of Pennsylvania Philadelphia, PA 19104-6371